<html>
<body>
<h2>Search Quality Benchmarking.</h2>
<p>
This package allows to benchmark search quality of a Lucene application.
<p>
In order to use this package you should provide:
<ul>
  <li>A <a href="../../search/Searcher.html">searcher</a>.</li>
  <li><a href="QualityQuery.html">Quality queries</a>.</li>
  <li><a href="Judge.html">Judging object</a>.</li>
  <li><a href="utils/SubmissionReport.html">Reporting object</a>.</li>
</ul>
<p>
For benchmarking TREC collections with TREC QRels, take a look at the 
<a href="trec/package-summary.html">trec package</a>.
<p>
Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:

<pre>
    File topicsFile = new File("topics-701-850.txt");
    File qrelsFile = new File("qrels-701-850.txt");
    Searcher searcher = new IndexSearcher("index");

    int maxResults = 1000;
    String docNameField = "docname"; 
    
    PrintWriter logger = new PrintWriter(System.out,true); 

    // use trec utilities to read trec topics into quality queries
    TrecTopicsReader qReader = new TrecTopicsReader();
    QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));
    
    // prepare judge, with trec utilities that read from a QRels file
    Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));
    
    // validate topics & judgments match each other
    judge.validateData(qqs, logger);
    
    // set the parsing of quality queries into Lucene queries.
    QualityQueryParser qqParser = new SimpleQQParser("title", "body");
    
    // run the benchmark
    QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
    SubmissionReport submitLog = null;
    QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);
    
    // print an avarage sum of the results
    QualityStats avg = QualityStats.average(stats);
    avg.log("SUMMARY",2,logger, "  ");
</pre>

<p>
Some immediate ways to modify this program to your needs are:
<ul>
  <li>To run on different formats of queries and judgements provide your own 
      <a href="Judge.html">Judge</a> and 
      <a href="QualityQuery.html">Quality queries</a>.</li>
  <li>Create sophisticated Lucene queries by supplying a different 
  <a href="QualityQueryParser.html">Quality query parser</a>.</li>
</ul>

</body>

</html>
